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WWC Study Reports are intended to support decision making; neither the What Works Clearinghouse 
(WWC) nor the U.S. Department of Education endorses any interventions. No single Study Report should 
be used as a basis for making policy decisions because (1) few studies are designed and implemented 
flawlessly and (2) all studies are tested on a limited number of participants, using a limited number of 
outcomes, at a limited number of times, so generalizing from one study to any context is very difficult. 

To highlight these issues, the WWC Study Reports describe in detail the specifics of each study, focusing 
primarily on studies that provide the best evidence of effects (randomized controlled trials). Systematic 
reviews of the evidence will be conducted to summarize the results of the individual studies. 

See the WWC Detailed Study Report (PDF) for more information about this study. 



Topic: Curriculum-Based Interventions for Increasing K-12 Math Achievement — 
Middle School 

Connected Mathematics Project 

Intervention: 



Research Design: 
Study Rating: 



Quasi-Experimental Design with Matching 

o 



Date Released: September 15, 2004 

Summary of Results: Schneider found no statistically significant differences between the 

intervention and comparison groups on the school-level Texas 
Assessment of Academic Skills (TAAS) analyzed in the study. The findings 
may have been affected by only three schools implementing more than 
one-third of the total math problems in each year and grade. In addition, 
the sample size was small. 






= Meets Evidence Standards 



= Meets Evidence Standards with Reservations 



Q 



= Does Not Meet Evidence Standards 



The What Works Clearinghouse (www.whatworks.ed.gov) was established in 2002 by the U.S. 
Department of Education’s Institute of Education Sciences to provide educators, policymakers, 
researchers, and the public with a central and trusted source of scientific evidence of what works in 
education. Please email all questions and comments to info@whatworks.ed.qov . The What Works 
Clearinghouse is administered by the U.S. Department of Education through a contract to a joint 
venture of the American Institutes for Research and the Campbell Collaboration . 



WWC Brief Study Report 



1 



What Is This Report About? 

The Connected Mathematics Project (CMP) is a 
comprehensive middle school mathematics 
curriculum developed for grades 6 through 8. 
The CMP is reported to align with the National 
Council of Teachers of Mathematics (NCTM) 
standards. This Brief WWC Study Report 
reviews a study of the effect of the CMP 
curriculum on students’ mathematics 
achievement on the Texas state assessment test. 
This report summarizes the study and reviews 
its strengths and weaknesses. 

How Was the Study Conducted? 

Schneider compares the mathematics 
performance of schools using the CMP 
curriculum with comparison schools on the 
Texas Assessment of Academic Skills (TAAS). 
Like the CMP curriculum, the TAAS — the 
mandatory state assessment test — is reported to 
align with the NCTM standards. Schneider did 
not report on what curriculum the comparison 
schools used. 

Schneider reports on schools selected from the 
pool of schools in Texas using the CMP 
curriculum. Another selection criterion was that 
teachers from these schools attended the six-day 
summer training organized by the Texas 
Statewide Systematic Initiative (SSI) in 1996, 
1997, and 1998. Twenty-three schools using the 
CMP curriculum were selected. Comparison 
schools in Texas were selected by matching them 
to the 23 schools using the CMP on the basis of 
the prior school-level passing rates on the TAAS, 
school enrollment, and demographic data such as 
school mobility and ethnicity. On the basis of 
these criteria, 23 schools were first selected for 
the comparison group. Two additional schools 
were added to the comparison group because 
they were feeder schools to two of the 
comparison schools already in the sample. Thus, 
there were a total of 25 comparison schools. 



Schneider reports two outcome measures of 
interest: the school-level TAAS passing rate in 
mathematics and the student-level Texas 
Learning Index (TLI). TLI is a TAAS-scaled 
score pegged to the exit-level passing standard. 
These school-level outcome measures are 
reported because their level matches the level of 
intervention delivery. 

Schneider groups the students into three groups. 
Students who experienced the CMP curriculum 
from grades 6 through 8 in 1996-1997 and 
1998-1999 are in Group 1. Students who had 
the curriculum in grades 6 and 7 from 1997- 
1998 and 1998-1999 are in Group 2. Students 
who had the curriculum only in grade 6 in 
1998-1999 are in Group 3. 

In addition, Schneider also groups a subset of 
schools that are defined as high-implementation 
schools or schools that reported covering more 
than one-third of the curriculum in one grade 
and in one year. None of the schools using the 
CMP curriculum report having the goal of 
completing the full curriculum in one grade and 
one year. 

What Did the Study Find? 

Schneider tests the significance of the TAAS 
passing rate findings through hierarchical linear 
modeling (HLM) analyses. The HLM analysis 
did not find any significant difference between 
schools using CMP and comparison schools. No 
significant difference is found in the analyses 
separated by groups of varying duration of 
intervention. In addition, no significant 
difference is found in analysis of high- 
implementation schools using CMP and the 
matched comparison schools. 

In a follow-up, Schneider also provides the TLI 
posttest mean scores at each school, allowing 
the means and standard deviations at the school 
level to be derived. Figures la to lc show the 
means and standard deviations for each group. 
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Figure la. Impact Reported by Schneider (2000): a School-Level TLI 
Posttest Scores in Group 1 
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Note. TLI = Texas Learning Index. 

a Confidence intervals are computed by the WWC. Schneider reports that the intervention group 
scores do not differ significantly from the comparison group scores on the Texas Assessment of 
Academic Skills. 
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Figure 1b. Impact Reported by Schneider (2000): a School-Level TLI 
Posttest Scores in Group 2 
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Note. TLI = Texas Learning Index. 

a Confidence intervals are computed by the WWC. Schneider reports that the intervention group 
scores do not differ significantly from the comparison group scores on the Texas Assessment of 
Academic Skills. 
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Figure 1c. Impact Reported by Schneider (2000): a School-Level TLI 
Posttest Scores in Group 3 
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Note. TLI = Texas Learning Index. 

a Confidence intervals are computed by the WWC. Schneider reports that the intervention group 
scores do not differ significantly from the comparison group on the Texas Assessment of 
Academic Skills. 



How Can You Find Out More? 

■ To learn more about this study, read the 

detailed report (PDF) . 

■ See reports on other studies of Middle 
School Math curricula. 

■ Cost information is available at the 
publisher’s website 

www.phschool.com/math/cmp/index.html 

■ Intervention Developer Contact 
Information: Contact the local Prentice Hall 
Sales Representative at their general number 
1-800-848-9500; contact CMP at 
517-432-2820; or visit the website 
www.math.msu.edu/cmp/index.html . 



Report Production 

■ Date created: September 15, 2004 

■ Topic area reviewed under: Curriculum- 
Based Interventions for Increasing K-12 
Math Achievement — Middle School 
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WWC Study Ratings 3 : Schneider (2000) 



Causal Validity: Meets WWC Standards with Reservations, a Quasi-Experimental Design with 

Matching and No Attrition or Disruption Problems 



Intervention schools were matched on relevant variables with similar comparison schools in Texas. The author does not 
address pretest differences between the two groups after the selection of the matched comparison schools. There was no 
attrition at the school level, and no extraneous events were identified that appeared to confound the intervention’s effect. 



Other Study 
Characteristics 


Study Rating 


Study-Specific Information 


Intervention Fidelity 


• • 


The CMP intervention is well implemented and well designed and meets the 
definition of middle school math. 


Outcome Measures 


• # 


The primary outcome measure (TAAS) shows evidence of face validity and 
reliability and is properly aligned. 


People, Settings, and 
Timing 


• 


Although the sample represents some important characteristics of targets, it does not 
represent others. The outcome measure was given some time in April, just before the 
end of the school year. 


Testing within 
Subgroups 


• 


The intervention effect was tested across the entire sample but not within important 
subgroups. 


Analysis 


• 


Schneider’ s analysis is at the school level, which is the same level as the unit of 
intervention delivery. The sample sizes are too small at the school level to allow for 
sufficiently precise estimates of the effect size. 


Statistical Reporting 


• # 


The statistical tests are adequately reported, and effect sizes can be estimated for the 
outcome measure of interest. 



Note. •• Fully meets criteria; • Meets minimum criteria; X Does not meet criteria. 

a For more information on the criteria used to rate this study, see the WWC Evidence Standards: Middle School Math . 
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